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[57] ABSTRACT 

A proactive interface management system for ascertaining a 
computer user's goals and automatically extracting 
untagged data from one or more source objects and applying 
the data to one or more target objects. The proactive inter- 
face management system includes several cooperating pro- 
gram elements including a user interface, a decision engine, 
an extractor, and an applicator. These cooperating program 
elements automatically perform the steps of recognizing 
user gestures, ascertaining user goals, identifying relevant 
source information in one or more source objects, extracting 
the relevant source information, and applying the extracted 
information to one or more target objects. 

23 Claims, 12 Drawing Sheets 
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From: Eric Horvitz 
To: Terri Hendry 
Ce: bbb@hpp.stanford.edu 
Subject RE: Visit to Microsoft 
Date: Monday, August 29,1994 12:57PM 

Let's assume that the talk will be from 10:30 
am-12:00 (with questions). It will take about 
45 minutes to get to MS from the airport 
You and i can meet at 9:30 or so to chat, get 
things set up. I think that you should make 
reservations for the evening flight out, to 
allow for meetings until 5pm. 
Eric 
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GOALr DRIVEN INFORMATION 
INTERPRETATION AND EXTRACTION 
SYSTEM 

TECHNICAL FIELD 

The present invention relates to information handling 
systems and more particularly relates to a method and 
system for extracting goal-related information from a source 
object and storing the extracted information in a target 
object 

BACKGROUND OF THE INVENTION 
There is a continuing need to develop user-friendly and 
powerful computer software. Causing the computer to auto- 
matically perform certain functions that previously required 
direct user interaction is one way in which software devel- 
opers can make software programs easier and more conve- 
nient to use. For example, configuration programs have been 
developed that automatically set-up a user's computer work- 
space upon system start-up. These configuration programs 
launch certain applications and open certain files in accor- 
dance with a user profile so that the user does not have to go 
through the process of manually launching die applications 
and opening the files. Another example of an automatically- 
performed computer function is a word processing program 
that sets a new document to the user's preferred font, type 
size, and page layout when the document is initially opened. 
This saves the user from the process of manually setting 
these parameters within the new document 

The preceding examples of system and document initial- 
ization are relatively simple examples of proactive interface 
management. There are many other more complicated situ- 
ations in which proactive interface management can be 
employed to save the user the time and frustration of having 
to perform certain steps manually. In these complicated 
situations, however, sophisticated logic may have to be 
employed to ascertain a user's goals and automatically take 
certain steps toward attaining those goals. For example, the 
process of extracting data from one file and inserting the data 
into another file through the usual "cut and paste* proce- 
dures can be tedious. A source program must be launched; 
a source file must be opened; the data to be extracted must 
be located and "cut"; a target program must be launched; a 
target file must be opened and the destination location 
identified; and finally the extracted data may be "pasted" 
into the destination location within the target file. 

These data transfer procedures can become time consum- 
ing and tedious if performed repetitively. In particular, many 
users have a recurring goal of extracting schedule-related 
data from electronic mail (e-mail) or posted on Internet web 
page and storing the extracted data in an electronic calendar 
file. For this type of recurring data extraction task, it would 
be advantageous if the computer system could ascertain the 
user's goal from a few simple commands, and then auto- 
matically perform the "cut and paste" operation without 
further input from the user. Even if the computer system 
cannot automatically complete the entire "cut and paste" 
operation, it would be helpful if the computer system 
performed part of the task and then allowed the user to easily 
complete the remaining portion of the task, for example by 
confirming or altering actions suggested by the computer 
program. The specific situation of extracting schedule- 
related data from e-mail, and storing the extracted data in an 
electronic calendar program is described in more detail 
below. 

Computer users are increasingly interconnecting their 
computers through networks, such as local area networks. 



2 

wide area networks, ethernets. and the Internet. Intercon- 
nected computer users typically communicate with each 
other through e-mail, which is a real-time, paper-free, cost- 
effective mode of communication that has become a stan- 

5 dard feature of virtually all computer networks. User- 
friendly, menu-driven e-mail application programs allow 
users to generate e-mail messages quickly and send them to 
large numbers of recipients. A recipient usually receives an 
e-mail message within seconds. A few simple key strokes or 

10 point-and-click operations allow the recipient to activate his 
or her e-mail application program and read the e-mail 
message. Specifically, an e-mail application program 
responds to these key strokes or point-and-click operations 
by storing the e-mail message in an e-mail file on the user's 

1 5 computer and displaying the e-mail message within an 
e-mail application window on the user's computer monitor. 

Computer users in workplaces that rely on e-mail may 
spend a lot of time reading, writing, and handling e-mail 
messages. A convenient and efficient system for handling 

20 e-mail messages may be critical to the productivity of these 
workplaces. For example, a received e-mail message may 
contain schedule-related information, such as a meeting 
notice, seminar announcement, lunch date, or the like. A 
computer user may also rely on an electronic calendar 

25 program to organize a personal schedule. A computer user 
may therefore have a need to transfer schedule-related 
information from an e-mail message to an electronic calen- 
dar file. For example, a computer user may receive an e-mail 
message containing a notice of a meeting that the user wants 

30 to attend. Upon reading the e-mail message, the user may 
need to record a notation of the meeting in an electronic 
calendar file. 

If the electronic calendar program is not resident in the 
computer system's memory, the user must first launch the 

35 calendar program and open a particular calendar file in order 
to transfer the schedule -related information from the e-mail 
file to the calendar file. Upon opening a particular calendar 
file, the electronic calendar program typically displays infor- 
mation from the calendar file within an electronic calendar 

40 application window on the user's computer monitor. Open- 
ing a calendar file thus causes the display of the e-mail 
application window to be replaced by the display of the 
electronic calendar application window. Once the calendar 
file has been open, the user typically must scroll to or 

45 otherwise locate the correct date in the calendar file so that 
the desired information from the calendar file is displayed 
within the electronic calendar application window. Once the 
information for the correct date is displayed within the 
electronic calendar application window, the user may have 

50 to refer back to the e-mail application window before 
entering a notation into the electronic calendar application 
window. Thus, the user may have to switch back and forth 
between the e-mail application window and the electronic 
calendar application window in order to correctly record the 

55 notation in the electronic calendar file. This process is slow, 
annoying, and presents the user with opportunities to make 
errors in transferring the information. 

It would therefore be advantageous to have a system to 
facilitate transferring goal-related information from a source 

60 file, such as an e-mail file, to a target file, such as an 
electronic calendar file. It would be particularly advanta- 
geous if the system could automatically interpret the source 
file so that the correct information could be automatically 
extracted and stored in the target file with minimal user 

65 involvement. This task may be difficult if schedule-related 
information is to be extracted from an e-mail file because an 
e-mail file typically includes information written in ftee- 



03/07/2003, EAST Version: 1.03.0007 



5,864,848 



text, such as prose. Interpreting free-text information is a 
cognitive function that is difficult to emulate with pro- 
grammed logic. 

More specifically, an e-mail message is typically dis- 
played within a semi-structured e-mail application window 
that includes a structured area and an unstructured area. The 
structured area of the e-mail application window typically 
includes a number of fields for predefined information, such 
as fields for *1o." 'from." "cc." "subject," and "date." The 
unstructured area of the e-mail application window typically 
includes free-text information written in prose. Furthermore, 
schedule-related information is usually embedded within the 
free-text information included in the unstructured area of the 
e-mail application window. In some cases, however, the 
structured fields may also contain schedule-related informa- 
tion or information that is helpful in interpreting certain 
free-text terms included in the unstructured area of the 
e-mail application window. 

The meaning of a particular message term in an e-mail file 
may therefore depend on the meaning of other terms within 
the e-mail file. Indeed, the meaning of a particular e-mail 
message term may depend on the context of the free-text 
portion of the message as a whole. Similarly, the meaning of 
message terms may vary from user to user depending on the 
lexicon used by authors of messages. Prior information 
handling systems do not handle the tasks of interpreting 
free-text information or adapting to a specialized lexicon 
very well. There is therefore a need for an improved method 
and system for interpreting source files, such as e-mail files, 
that contain free-text information. It would be advantageous 
for the improved method and system to include an adaptive 
lexicon, in which the logic used to interpret free-text infor- 
mation may be tailored in response to source files previously 
interpreted for a particular user. 

The needs identified above are described in the context of 
a system in which the goal is extracting schedule-related 
information from an e-mail file and storing the extracted 
information in an electronic calendar file. A similar need 
exists, however, for many different goals for which infor- 
mation may be extracted from a free-text source file and 
stored in a predefined format in a target file. For example, a 
sales manager may have a need to extract sales statistics 
from free-text sales reports and stare them in a database. 
Similarly, a researcher may have a need to extract informa- 
tion related to a particular research topic from free-text files 
within an electronic library, such as "LEXIS," and store the 
extracted information in a specialized format. 

Generally stated, there is a need for an improved method 
for ascertaining a user's goals and automatically taking 
certain steps toward attaining those goals. Specifically, there 
is a need for a proactive interface management system that, 
in recognition of user goals, automatically identifies and 
extracts relevant source information from one or more 
source objects for application to one or more target objects. 

More specifically, there is a need for an improved method 
and system for interpreting a source file containing free-text 
information, identifying goal-related terms, and extracting 
the goal-related terms for storage in a target file in a 
predefined format There is also a need for ensuring that the 
source file has been properly interpreted prior to recording 
the extracted information in the target file. There is also a 
need for an information interpretation and extraction system 
with an adaptive lexicon, in which the logic used to interpret 
free-text information is tailored in response to source files 
previously interpreted for a particular user. 

SUMMARY OF THE INVENTION 

The present invention is a method and system for ascer- 
taining a computer user's goals and automatically taking 



certain steps toward attaining those goals. A proactive inter- 
face management system according to the present invention 
allows data to be extracted from one or more source objects 
and applied to one or more target objects. The proactive 
5 interface management system includes several cooperating 
program elements including a user interface, a decision 
engine, an extractor, and an applicator. These cooperating 
program elements automatically ^ejfojm»thc*stepSTU3fi^G^ 
ogmziflg-useD^e*tu1es?*a^^ 
10 relevant source information in one or more source objects, 
extracting the relevant source information, and applying the 
extracted information to one or more target objects. 

According to an aspect of the present invention, a user 
goal is ascertained by inferring a user gflrd-ba'ind»on-r^ 
15 piedefined^^^^ftm^^T^^^T^^y 1 ^ »ftf «1T f TU™mmanf1 fi ■ ■ > 
r ec^iyjed^ Airinfl^ If the 



likelihood of the goal inferred does not exceed a threshold, 
then confirmation of an action may be requested of the user 
or the user may be presented with a list of alternative actions 
20 for selection. 

Once the goal has been selected, the extractor of the 
present invention implements interpretation and extraction 
logic that extracts the desired information from the source 
object, including untagged portions of the source object, for 
25 inclusion in the target object. In order to implement the 
selected goal. tite ^cbc ut'ln vcirtiwttsewn^ 
oj^ efoto^intenyefr in f*^ 

r^tiftfttfr^^^t^rcgKg^jgct Each template that extracts 
information from untagged portions of a source file includes 
a lists of expected data patterns organized under a system of 
tokens. Each token is tied to one or more target object fields 

in an application-specific template. 

ug source "ODje'ct 



30 



1 



data^paaerasjdoaihe^expecte^ th e 
jcmplateF^ mm 

The application-specific template containing the set of 
target object fields is displayed to the user for review and 
modification. The user may quickly complete and/or correct 
the target object fields and apply the completed and/or 
corrected data to the target object The-*texteon-of-the- 
xemrjdatfoMfigQ^to generate the set of target object fields for 
appScauon to the target object may be updated with the 
completed and/or corrected information input by the user. 
This allows the lexicon of the application-specific template 
to be updated with information learned from manual user 
input 

In view of the foregoing, it will be appreciated that the 
present invention provides an improved method and system 
for interpreting a source object containing untagged 
information, identifying goal-related information, and 
extracting the goal-related information for application to a 
target object. TTiat the present invention improves over the 
drawbacks of the prior art and provides the advantages 
described herein will become apparent from the following 
detailed description of the disclosed embodiments and the 
appended claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

go FIG. 1 is a functional block diagram of a personal 

computer system interconnected with a network. 
FIG. 2 is a diagram illustrating the software architecture 

of a proactive interface management system according to the 

present invention. 
65 FIG. 3 is a logic flow diagram illustrating the operation of 

a proactive interface management system according to the 

present invention. 
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FIG. 4 is a diagram illustrating prior art user interfaces manipulated. It is conventional for those skilled in the art to 

including an e-mail application window and an electronic refer to these signals as bits, bytes, values, elements, 

calendar application window. symbols, characters, terms, numbers, records, files, or the 

FIG. S is a diagram illustrating a user interface displaying ** c - » should be kept in mind, however, that these and 

an autoscheduler template window and an e-mail application 5 similar terms should be associated with appropriate physi- 

window in accordance with the present invention. cals quantities for computer operations, and that these terms 

, . . _„ - A . .. are merely conventional labels applied to these physical 

FIG. 6 compnsmg WGS-6A-6C *™ » a PP u - quantities that exist within the computer, 

cation window, an autoscheduler template window, and an understood that the programs, 

electronic calendar application window. ' J A , , rtr . 

^ r , 10 processes, methods, etc.. described herein are not related or 

FIG. 7 is a logic flow diagram showing the steps of a tQ computer, single chip processor, or 

computer-implemented process for an illustrative embodi- a p para tus. Rather, various types of general purpose 

ment of a goal-driven information interpretation and extrac- may be used with programs constructed in accor- 

tion system, the electronic mail auto scheduler. wffll the teacnings described herein. Similarly, it may 

FIG. 8 is a logic flow diagram showing the steps of a ^ prove advantageous to construct specialized apparatus to 
computer-implemented process for ascertaining the goal in perform the method steps described herein by way of 
the illustrative embodiment of a goal-driven information dedicated computer systems with hard-wired logic or pro- 
interpretation and extraction system, the electronic mail grams stored in nonvolatile memory, such as read only 
autoscheduler. memory. 

FIG. 9 is a logic flow diagram showing the steps of a 20 Exemplary Operating Environment 

computer-implemented process for automatically assigning Turning now to the drawings, in which like numerals 

data to an application-specific template in a goal-driven indicate like elements throughout the several figures, FIG. 1 

information interpretation and extraction system. shows an operating environment for the present invention 

FIG. 10 is a logic flow diagram showing the steps of a including a computer system 8 connected to a network 9. 

computer-implemented process for ranking candidate « The computer system 8 may be any of a variety of conven- 

tagged terms for assignment to target data fields of an ^al personal computer or similar work stations, including 

appUcation-specific template for a goal-driven information but not limited to "APPLE" 4 TBM" or "BM^-compatible 

interpretation and extraction system. personal computers including those known as "AT or 

t a A - , . ^ f "PS/2" compatibles. The network 9 may be any of a variety 

FIG. 11 is a logic flow diagram showing the steps of a * Jr? * * ^ • i J- u„* 

A . . 6 . ^ c f. . r . , . 30 of compatible network configurations, including but not 

compmer-implemented process for presenting target data .. . 7\ , , ^ f „ Axn . * . 

_ ,7 - „ , . • , • limited to a local area network (LAN)* wide area network 

fields of an application-specific template in a display win- _ 4 . _ * ti.. „^„~i, o 

. r K . j-c * i (WAN), ethernet. the Internet, or the like. The network 9 

dow to a user for review and modification in a goal-dnven ^ } * " 0 ... 

. A . , . _ interconnects the computer system 8 with one or more other 

information interpretation and extraction system. _ _ j ■ ♦ ^ „, r 

r J computer systems and may interconnect the computer sys- 

FIG. 12 is a logic flow diagram showing the steps of a 35 tcm g with one or morc peripheral devices, such as printers, 

computer-implemented process for saving the results in a modems< ^ serV ers, etc. The computer system 8 and the 

goalndriven information interpretation and extraction sys- othef computcr systcms interconnected with the computer 

tem * system 8 by way of the network 9 communicate with each 

DETAILED DESCRIPTION ower us ^ n 8 electronic mail (e-mail). The functions and 

40 operations of conventional e-mail are well known and will 

The present invention is directed toward a proactive not be further described herein, 

interface management system that makes complex data The computer system 8 includes a processing unit 12 

search and extraction tasks easier to perform. In one interconnected by way of a system bus 15 with a system 

embodiment, the invention provides an "autoscheduler" that random access memory (RAM) 16 and internal I/O devices 

extracts schedule related information from e-mail messages 45 including a CD-ROM drive 14, a hard disk drive 18, and a 

and applies the extracted information to electronic calendar floppy disk drive 20. The drives and their associated 

files. computer-readable media provide nonvolatile storage of 

The following detailed description is presented largely in computer readable instructions, data structures, program 

terms of processes and symbolic relationships of operations components and other data for the computer system 8. 

of data bits manipulated by a processing unit and maintained 50 Although the exemplary environment described herein 

within data structures supplied by one or more memory employs a hard disk, a removable floppy disk and a remov- 

storage devices. These data structures impose a physical able CR-ROM disk, it should be appreciated by those skilled 

organization on the collection of data bits stored within in the art that other types of computer readable media which 

computer memory and represent specific electrical or mag- can store data that is accessible by a computer, such as 

netic elements. These descriptions and symbolic represen- 55 magnetic cassettes, flash memory cards, digital video disks, 

tations are the means used by those skilled in the art of Bernoulli cartridges, random access memories (RAMs), 

computer programming and computer construction to most read only memories (ROM), and the like, may also be used 

efficiently convey teachings and discoveries to others skilled in the exemplary operating environment The system bus 15 

in the art. also interconnects the processing unit 12 with one or more 

For the purpose of this discussion, a method or process is 60 I/O ports 22. The processing unit 12 is typically iraple- 

generally conceived to be a sequence of computer-executed mented as a microprocessor, such as the 80386. 80486. or 

steps leading to a desired result. These machine- the various "PENTIUM" models manufactured by Intel 

implemented steps, which may constitute a program module. Corporation, Santa Clara, Calif. The I/O ports 22 are used to 

generally require physical manipulations of physical quan- interconnect the computer system 8 with a conventional user 

tities. Usually, though not necessarily, these quantities take 63 input/output system 23 including a keyboard 24. a mouse 25. 

the form of electrical, magnetic, or optical signals capable of a monitor 26, and a speaker 27. The I/O ports 22 are also 

being stored, transferred, combined, compared, or otherwise used to interconnect the computer system 8 with a network 
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connection port 28. The network connection port 28 is used information in a personal calendar file (the target file). The 

to interconnect the computer system 8 with other computers autoscheduier program 38 facilitates the handling of 

and peripheral devices via the network 9. schedule-related information received via e-mail. 

The system memory 16 includes an operating system 32. When an e-mail message contains schedule-related 

such as "WINDOWS 95" manufactured by Microsoft 5 information, such as information pertaining to a meeting that 

Corporation. Redmond. Wash. The operating system 32 the user would like to attend, the user may have a need to 

comprises a set of computer program instructions that con- record information from the e-mail file in another electronic 

trol the internal functions of the computer system 8 and file, such as a personal calendar file. Similarly, a user may 

thereby allow the computer system 8 to run application wish to extract other types of information from e-mail files 

software and other program modules. The operating system 10 for storage in other electronic files. For example, a sales 

32 is typically installed in a mass storage device, such as the manager may receive e-mail messages including sales 

CD-ROM 14. the hard disk drive 18. the floppy disk drive reports from sales associates. The sales manager may wish 

20, or a ROM chip (not shown). During boot up to extract sales statistics from these sales reports and store 

(initialization) of the computer system 8. the operating them in a database. It is further noted that the source file 

system 32 is loaded into the system memory 16. 15 containing untagged information such as a free-text message 

During operation of the computer system 8. application need not be an e-mail message. The principles of the present 

programs operate in conjunction with operating system 32 to invention may be applied to any type of electronic source file 

perform specialized functions. Common application pro- that contains untagged information. For example, the present 

grams include word processors, spread sheets, databases. invention could be used to search electronic information 

games, etc. In performing these specialized functions, the 20 systems, such as "LEXIS" or other generally available 

application programs send information to the operating electronic information sources to extract address informa- 

system 32. from which the operating system creates appli- tion for storing in a database. Furthermore, those skilled in 

cation windows for visual presentation to a user on the the computer programming arts will appreciate that 

monitor 26. autoscheduier program 38 described herein may be modified 

Certain executable program modules associated with an 25 to perform information interpretation and extraction for 

application program must be loaded into the system memory other goals. 

16 before the application program can run on the computer In connection with the disclosed autoscheduier embodi- 
system 8. Launching an application program includes load- ment of the present invention, a first application program 
ing the executable program modules necessary to run the running on the computer system 8 is an e-mail program 36, 
application program into the system memory 16 of the 30 which allows the computer system 8 to receive and send 
computer system 8. The application programs are typically e-mail messages from or to other computer systems on the 
stored in a mass storage device, such as the CD-ROM drive network 9. A second application program running on the 
14, the hard disk drive 18. or the floppy disk drive 20 until computer system 8 is a specialized autoscheduier program 
a user initially launches the application program after boot- 38. The autoscheduier program 38 is described with more 
ing up the computer system 8. Several application programs 35 particularity with respect to FIGS. 3-11. A third application 
may be simultaneously loaded into the system memory 16 of program running on the computer system 8 is a conventional 
the computer system 8. A user may then switch quickly electronic calendar program 40. The e-mail program 36. the 
among the various application programs. A user typically autoscheduier program 38. and the electronic calendar pro- 
selects a particular application program for execution by gram 40 may all be simultaneously stored in system memory 
positioning a pointer on an icon or window associated with 40 16 for convenient access by the user, 
the application program and dieting a mouse button. For DESCRIPTION OF THE DISCLOSED 
example, a user typically launches the e-mail program 36 E^ODIMEOTS 
upon boot up so that the user can quickly activate the e-mail 

program 36 by clicking on an icon, menu item, or window FIG. 2 is a diagram illustrating the software architecture 
anytime throughout the day. Typically, a visual cue or an 45 of a proactive interface management system 200 according 
audible sound may be activated whenever a new e-mail to the present invention. Generally stated, the proactive 
message arrives to alert the user that a new message is interface management system 200 monitors the user corn- 
waiting in his or her mailbox. This makes using the e-mail mands 202 and the system context 204 to make complex 
program 36 convenient throughout the day. data search and extraction tasks easier to perform. The user 
The present invention is a method and system for extract- 50 commands 202 include all of the conventional commands 
ing goal-related information from a source object that that a user may communicate using the user input/output 
includes untagged information and storing the extracted system 23. The system context 204 includes the configura- 
information in a target object The source object may be a tion computer system 8. as well as the current state of the 
file. The target object is typically a special purpose file for various hardware and software elements within the corn- 
handling information of the type extracted from the source 55 puter system 8. For example, the system context 204 
file. A goal in this context is the identification and extraction includes status information regarding the program modules 
of information pertaining to a predefined subject A source that are loaded in the system memory 25. the extensions that 
file is an electronic file including untagged information, such are active, the servers that are accessible to the system, the 
as free-text information, which may have been created with program modules that are available via the servers, etc. 
a word processor or scanner. A target file is an electronic file 60 The proactive interface management system 200 pref er- 
ror handling extracted information in a predefined manner. ably includes four cooperating computer software 
such as an electronic database or other special purpose components( which may be implemented as separate objects 
information handling file. For example, a disclosed embodi- in an object-oriented software environment): a user interface 
ment of the present invention referred to herein as an 206. a decision engine 208, an extractor 210. and an appli- 
"autoscheduler" provides a method and system for identi- 65 cator 214. These elements allow data to be extracted from 
fying and extracting schedule-related information (the goal) one or more source objects 212a~2\ln and applied to one or 
from an e-mail file (the source file) and storing the extracted more target objects 216a-216m. Each element performs 
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certain functions and interacts with other elements through 
predefined object-oriented interfaces, as described below. 
Those skilled in the art will appreciate the division of 
functions among these software dements, and the interfaces 
between these elements, may be altered somewhat within 
the scope of the present invention. For example, a function 
could be reassigned from one element to another, two or 
more of these elements could be combined into a single 
element a single element could be divided into two or more 
cooperating elements, etc. In the disclosed autoscheduler 
embodiment of the present invention, the autoscheduler 
program contains the functionality of all four of these 
components. 

The proactive interface management system 200 causes 
the computer system 8 to perform sophisticated data extrac- 
tion tasks automatically or with a ininimum amount of user 
interaction. Pop-up windows 218 are preferably employed 
for user interaction at irnportant, points in the data extraction 
process. The use of pop-up windows allows the proactive 
interface management system 200 to query the user for 
specific information, and to confirm instructions, with a 
minimum of user action. This allows the user's interaction to 
be focused on narrow instructions in short intervals. 

High-level user interface commands are precursors to the 
proactive interface management system 200. For example, 
as an extension of the common notion of "drag and drop" 
implemented in many PC operating systems, an icon repre- 
senting a source object may be selected, dragged across the 
computer screen, and dropped on another icon representing 
a target object A goal is inferred based upon the object types 
involved in this "drag and drop** combination to perform a 
simple data extraction task that the computer system 8 
performs automatically in response to a "drag and drop" user 
gesture. The proactive interface management system 200 
extends the conventional "drag and drop" operation to 
provide a sophisticated and dynamic set of data extraction 
tasks that can be performed by the computer system 8 in 
response to relatively simple user gestures. 

The data extraction tasks performed by the proactive 
interface management system 200 differ from a conventional 
"drag and drop" operation in that the decision engine 208 
allows the user's goal-to extract a particular type of data 
from one or more source objects and store the extracted data 
in one or more target objects — to be ascertained in a wide 
variety of ways. In addition, the extracted data can be 
maintained in a wide variety of formats such as free-text 
pictorial bit maps, video, sound, etc. Moreover, the extractor 
210 may interpret the data within one or more source 
objects, identify a relevant portion of the data, and extract 
only the relevant portion of the data for storage in one or 
more target objects. As noted previously, the proactive 
interface management system 200 may use a pop-up win- 
dow at virtually any point in the data extraction process to 
gain additional information from the user, or to confirm a 
proposed course of action. 

The user interface 206 performs the function of recog- 
nizing user gestures that indicate data extraction tasks. A 
user gesture may be any type of explicit command that 
associates one or more objects with a data extraction 
command, such as a conventional "drag and drop" gesture. 
Another type of explicit user gesture might involve focus- 
ing" on one or more objects, such as a source object and a 
target object and then clicking on a menu item or entering 
a predefined command sequence. Alternatively, a selection 
window may be displayed on the monitor 26 that allows the 
user to enter or select one or more source files, one or more 
target files, and one or more data extraction commands. 
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Those skilled in the art will appreciate that many other types 
of explicit commands may be recognized from the user 
input-output system 23. 
In addition, the user interface 206 may be programmed to 

5 recognize certain implicit user gestures. For example, the 
user interface 206 may be selectively programmed to rec- 
ognize an implicit user gesture whenever the user downloads 
a file that contains a sound component. That is. the user 
interface 206 may automatically interpret the downloading 

10 of a file including a sound component as a command to 
extract the sound component from the file for storage in a 
target file, such as the user' s library of sound files. This type 
of implicit gesture may be useful for searching a distributed 
information source, such as the Internet for files including 

15 sound components, downloading files, and extracting the 
sound components to build an archive of sound fiies.T* 
s^^ *M5A^rlHttfld B tJ t U a^t^pmtiuu ' tuui^Kx^ifw u^d' f oT 
pj^eB al»b i t«i nflpfc8^id ^^ 

20 A user gesture indicating a data extraction command 
typically defines one or more source objects and/or one or 
more target objects. It will be understood that the recogni- 
tion of a user gesture indicating a data extraction command 
initiates a sophisticated data analysis and extraction process. 

25 Therefore, a pop-up user window is typically utilized to 
query the user to confirm a detected gesture before the user 
interface 206 proceeds. The pop-up user window may also 
be utilized to query the user to provide additional informa- 
tion related to the data extraction command, such as the 

30 definition of the one or more source objects, one or more 
target objects, and/or one or more data extraction com- 
mands. 

Hie user interface 206 is preferably implemented as a 

33 dynamic link library (DLL) that may be invoked either by 
the shell of the operating system or by an application 
program that is manipulating a particular source file. The 
user interface 206 monitors the user commands 202 and 
recognizes certain user gestures, as described above. Upon 

^ recognizing a user gesture, the user interface 206 calls the 
decision engine 208. which further processes the user's 
command. Like the user interface 206, the decision engine 
208 is preferably implemented as a DLL. 
The decision engine 208 may use a wide variety of 

45 decision-making techniques to ascertain the user's goal. For 
example, the decision engine 208 may respond to explicit 
pre-defined user commands, such as command key 
sequences or combinations. Alternatively, the decision 
engine 208 may utilize one or more complex decision - 

50 making techniques such as a Bayesian inference network or 
a deterministic rule- based algorithm. In addition, a pop-up 
user window may be utilized to query the user to provide 
additional information for the decision-making process, or 
to confirm a goal proposed by the decision engine 208. A 

55 pop-up user window may also be used to confirm an inferred 
goal before the decision engine 208 proceeds. Once the 
user's goal has been ascertained, the decision engine 208 
calls the extractor 210. and typically passes the extractor 210 
a pointer to one or more source objects and/or one or more 

50 target objects. 

The extractor 210 may use a wide variety of techniques to 
identify special attributes within the source information. For 
example, extractor 210 may identify bit maps if the user's 
goal involves extracting bit maps, or sound elements if the 

63 user's goal involves extracting sound elements, or words or 
symbols pertaining to a particular topic if the user's goal 
involves extracting text related to that topic, etc. For free- 
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text portions of text files, a Boolean instruction may be used fields within the electronic calendar file. For example, the 

to identify text patterns or combinations. Atenativdy^fcs autoscheduler may use the template to tag the terms "today " 

tQmphitesofseie^aafctefms^Hfi^ 'tomorrow." u next Friday," etc. with the token "day word." 

sa2ioas=o£3auixe2otye^^ The "day word" token corresponds to the "date" target field 

subject^=^ 5 in the electronic calendar file under which the extracted 

The extractor 210 may also determine which portions of information will ultimately be stored. The extractor includes 

the source information are relevant to the user's goal. Some an application -specific rule base with each application- 

of the extracted information may be parsed from structured specific template. Ea ehMRil&*tar s c p faranfiy^ 

fields of a source object, such as the sender name and subject utfe&jawtkig^eessa^e^HHS^ message-tErms^ 

fields of a source e-mail message. But other information may 10 to-toget^fiejd^ 



have to be extracted from the untagged portion of the source examplfejtharjd^^ 

object or from other objects that are linked or attached to the mcama ^^ m^^ 

source object. This step may involve any of a wide variety Ue^ajid=tiMitfe^^ 

of sophisticated techniques for identifying relevant data Once the candidate terms are identified by tagging, then 

within the source objects. For a user goal of extracting bit 15 they are prioritized in descending order of likelihood for 

maps pertaining to a particular type, correlations, projection extraction or presentation to the user. In the illustrative 

histograms. Hough transformations, and other image analy- embodiment of the autoscheduler, the candidate terms are 

sis techniques may be used to determine whether the source ranked through a two step process of first grouped them into 

files include bit maps containing the desired types of images. clusters to build sets of candidate fields. Then the clusters are 

For a user goal of extracting music, cadence analyses may 20 ranked in priority order. A tagged term is grouped into a 

be performed to determine whether the source files include cluster based on its spatial relationship to other tagged terms 

sound elements containing music. in mc sourcc file- For example, all tagged terms that are 

One of the more challenging tasks involves the interpre- within four sourcc mc tajas of anothcr terra may be 

tation of free-text portions of source objects. Interpreting int0 mc ^ster. As stated above, once the 

free-text information may involve the use of one or more 25 candidate terms have been grouped in clusters, the clusters 

predefined templates of terms that are relevant to the user's m rankcd * 11 has becn discovered that clusters that contain 

goal. The extractor 210 typically applies such a template to more relevant terms are generally more likely to include 

the source object to tag candidate terms that are most likely important information, and that clusters in which the rel- 

relevant to the user's goal. The candidate terms are then evant terras «* mofe & ou !* d «* generally more 

prioritized in descending order of likelihood for extraction 30 10 include ^P 0 ** 0 * information. The preferred 

or presentation to the user. For example, the candidate terms method for rankin g >** clusters therefore assigns a higher 

may be grouped into clusters based on their spatial relation- priority to clusters that contain a higher number of expected 

ship to each other using a Bayesian, single linkage, complete terms - n * preferred ranking method also assigns and a 

linkage, or other type of clustering technique. Alternatively, P nontv to clusters that include terms that are closer 

certain terms of the template may be pre-ranked in descend- 35 together. More specifically, the following formula may be 

ing order of likelihood prior to clustering. The candidate used t0 rank clu sters: 
terms may then be grouped into clusters based on their 
spatial relationship to the highest priority pre-ranked terms 

that are tagged in the source object In addition, each goal k _ } 

may have an associated set of rules that are used in the 40 
prioritizing process, such as terms that are only tagged if 
they appear in certain phrases, terms that are assigned a 
higher probability if they appear following a date term, 
terms that are automatically replaced by synonyms, etc. 



cluster value = ?ta;-a*il 
1 



Many other types of cluster and ranking techniques may be 45 wncrc k is the number of relevant terms in the cluster and a, 

used to interpret free-text information in connection with the | s the number of source file terms from the relevant term(i) 

disclosed embodiments. to the beginning of the source file. The exponent 4 *2" may be 

An illustrative embodiment of interpreting free-text is varied, 

described below in the context of an "autoscheduler" that The highest ranking tagged terms are then inserted into 

extracts schedule related information from e-mail messages, so structured fields of a target file. For the autoscheduler 

That is, the principles of the present invention may be used application, the target fields typically include "date of 

to construct an autoscheduler for extracting schedule-related appointment" (e.g.. day. month, year), "start time of 

information (the goal) from an e-mail file (the source object) appointment." "end time of appointment." "location of 

and storing the extracted information in an electronic cal- appointment." "subject of appointment." and "source of 

endar file (the target object). For this application, the 55 schedule information." For an address extraction 

autoscheduler's application-specific template includes a application, the target file may include structured fields such 

"date" target field for identifying the date of a meeting, a as "name." "title." "street address 1," "street address 2." 

"time" target field for identifying the starting and ending "cityAown." "state, country." 'telephone number." 'fax 

times of the meeting, a "place" target field for identifying the number." "e-mail address." and *Veb home page." It will be 

location of the meeting, and an "attendees" target field for 60 appreciated that many other types of target files may be 

identifying the meeting attendees. The information assigned designed corresponding to different extraction goals, 

to the target fields of the autoscheduler template may be Because the data analysis performed by the extractor 210 

saved to create an entry in the target calendar file. typically requires a substantial allocation of computer 

Expected terms (i.e., schedule-related terms) within the resources, a pop-up user window is typically utilized to 

e-mail file are tagged and assigned to target fields of the 65 query the user to confirm a proposed analysis before the 

autoscheduler* s application- specific template. The target extractor 210 proceeds. The pop-up user window may also 

fields of the autoscheduler template correspond to target be used to further define the data extraction task, and to add 
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or subtract from the list of source objects and target objects 
on which the extractor 210 will operate. 

Once the relevant data has been identified, the extractor 
210 calls the applicator 214 to apply the extracted data to the 
target objects. Like the extractor 210. the applicator 214 may 
be implemented as a DLL. Similar to the extractor 210. the 
applicator 214 may use a pop-up window to gain additional 
information from the user or to confirm a proposed data 
extraction task before proceeding. 

It will be appreciated that the proactive interface man- 
agement system 200 may be used to facilitate many different 
types of data search and extraction tasks. For example, bit 
maps may be extracted from source files on the Internet and 
stored in the user's library of bit maps. Similarly, music 
elements may be extracted from source files on the Internet 
and stored in the user's library of music files. Free-text 
pertaining to a particular topic may be extracted from source 
files on the Internet and stored in a target file on the user's 
computer system 8 for further review and analysis by the 
user at a later time. It will be appreciated that many other 
data extraction tasks may be performed by the proactive 
interface management system 200. Indeed, the power of the 
searching and data extraction engine provided by the pro- 
active interface management system 200 is limited only by 
the skill of the programmer in defining the above-described 
tools for the user, and the user's skill in applying these tools. 

FIG. 3 is a logic flow diagram illustrating a computer- 
implemented process 300 for the proactive interface man- 
agement system 200 according to the present invention. In 
step 302, the user interface 206 monitors the system context 
In step 304. the user interface 206 receives user commands 
and. in decision step 306. the user interface 206 determines 
whether a user gesture indicating a data extraction command 
is recognized. 

If a user gesture indicating a data extraction command is 
not recognized, the "no** branch is followed from step 306 
back to step 302. in which the user interface 206 monitors 
the system context If a user gesture indicating a data 
extraction command is recognized, the "y cs " branch is 
followed from step 306 to step 308. in which the user 
interface 206 calls the decision engine 208. In step 310. the 
decision engine 208 ascertains the user's goal for the data 
extraction command. In step 312. once the decision engine 
208 has ascertained the user* s goal, it calls the extractor 210. 

In step 314, the extractor 210 identifies the source infor- 
mation within the source objects, and in step 316 the 
extractor 210 identifies special attributes within the source 
information. In step 318, the extractor 210 determines which 
portions of the source information are relevant to the user's 
goal Once the relevant data has been identified, the extrac- 
tor 210 extracts the relevant source information in step 320. 
In step 322. the extractor 210 calls the applicator 214. and 
typically passes the applicator 214 a pointer to the extracted 
data and one or more target objects. In step 324, the 
applicator 214 applies the extracted data to the target 
objects. Alternatively, the extractor 210 may skip step 320 
and pass the applicator 214 a pointer to the relevant portions 
of one or more source objects and/or one or more target 
objects. In this case, in step 324, the applicator 210 extracts 
the relevant portions of the source objects and applies the 
extracted data to the target objects. 

Step 324 is followed by decision step 328, in which it is 
determined whether the user has indicated that the proactive 
interface management system 200 should quit upon com- 
pleting the data extraction task. If the proactive interface 
management system 200 has been instructed to quit, the 
"yes" branch is followed to the "end" step 330. and the 



14 

computer-implemented process illustrated by FIG. 3 is com- 
pleted. If. on the other hand, the proactive interface man- 
agement system 200 has not been instructed to quit, the "no" 
branch is followed to step 302, and the computer- 

5 implemented process illustrated by FIG. 3 repeats. 

In view of the foregoing, it will be appreciated that the 
proactive interface management system 200 may be used to 
facilitate many different types of data search and extraction 
tasks. Those skilled in the art will appreciate that searching 

10 and data extraction techniques appropriate to any particular 
task may be identified and made available to a user using the 
proactive interface management system 200. One specific 
example of the proactive interface management system 200 
that may be defined using the above-described technology is 

15 an "autoscheduler" that extracts schedule related informa- 
tion from e-mail messages. This particular example is par- 
ticularly instructive because it teaches an illustrative method 
for addressing the challenging task of interpreting free-text 
data in source objects. From this example, those skilled in 

20 the art will appreciate that powerful extractors may be 
defined for myriad other goals that may be defined by a user. 
Autoscheduler— An Illustrative Einbodiment 

As noted above, the proactive interface management 
system 200 described above may provide a method and 

25 system for extracting goal-related information from a source 
object, for this embodiment a source file, that includes 
free-text information. The extracted data may then be stored 
in a particular target object, for this embodiment a target file, 
such as a special purpose file for handling information of the 

30 type extracted from the source file. A goal in this context is 
the identification and extraction of information pertaining to 
a predefined subject. A source file is an electronic file 
including untagged information, such as free-text which 
may commonly be created with a word processor or scanner. 

35 A target file is an electronic file for handling extracted 
information in a predefined manner, such as an electronic 
database or other special purpose information handling file. 
For example, an illustrative embodiment of the present 
invention referred to herein as an "autoscheduler" provides 

40 a method and system for identifying and extracting 
schedule-related information (the goal) from an e-mail file 
(the source file) and storing the extracted information in a 
personal calendar file (the target file). The autoscheduler 
program 38 facilitates the handling of schedule-related 

45 information received via e-mail. 

When an e-mail message contains schedule-related 
information* such as information pertaining to a meeting that 
the user would like to attend, the user may have a need to 
record information from the e-mail file in another electronic 

50 file, such as a personal calendar file. Similarly, a user may 
wish to extract other types of information from e-mail files 
for storage in other electronic files. For example, a sales 
manager may receive e-mail messages including sales 
reports from sales associates. The sales manager may wish 

55 to extract sales statistics from these sales reports and store 
them in a database. It is further noted that the source object 
containing untagged information need not be an e-mail 
message. The principles of the present invention may be 
applied to any type of electronic source file containing 

60 untagged information. For example, the present invention 
could be used to search electronic information systems, such 
as "LEXIS" or other generally available electronic informa- 
tion sources to extract address information for storing in a 
database. Furthermore, those skilled in the computer pro- 

65 gramming arts will appreciate that autoscheduler program 
38 described herein may be modified to perform information 
interpretation and extraction for other goals. 
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In connection with the disclosed autoscheduler embodi- 
ment of the present invention, a first application program 
running on the computer system 8 is an e-mail program 36. 
which allows the computer system 8 to receive and send 
e-mail messages from or to other computer systems on the 
network 9. A second application program running on the 
computer system 8 is a specialized autoscheduler program 
38. The autoscheduler program 38 is described with more 
particularity with respect to FIGS. 5-13. A third application 
program running on the computer system 8 is a conventional 
electronic calendar program 40. The e-mail program 36. the 
autoscheduler program 38, and the electronic calendar pro- 
gram 40 may all be simultaneously stored in system memory 
16 for convenient access by the user. 

FIG. 4 is a diagram illustrating prior art user interfaces 
including an e-mail application window 43 and an electronic 
calendar application window 53. Prior to the present 
invention, the procedures required to extract schedule- 
related information from an e-mail file 41 and to store it in 
another file, such as a calendar file 54, were cumbersome. 
Those skilled in the art will appreciate that each application 
program running on the computer system 8 typically must be 
separately launched. Launching an application program can 
take a period of time ranging from a few seconds to half a 
minute or so to complete. Therefore, a user wishing to record 
information from the e-mail file 41 in the calendar file 54 
had to first read the e-mail message 42 within the e-mail 
application window 43. and then launch electronic calendar 
program 40. Once calendar program 40 was launched, the 
user had to open the desired calendar file 54 and then scroll 
or otherwise move through the calendar file 54 so that the 
information corresponding to the correct date was displayed 
within the electronic calendar application window 53. The 
user could then manually transfer the schedule-related infor- 
mation displayed in the e-mail application window 43 into 
the electronic calendar application window 53. 

Transferring the schedule-related information from the 
e-mail application window 43 to the electronic calendar 
application window 53 was difficult because typically one 
application window could be displayed on the monitor 26 at 
a time. Thus, opening the calendar file 54 caused the display 
of the e-mail application window 43 to be replaced by the 
display of the electronic calendar application window 53. 
The user therefore had to switch back and forth between the 
e-mail application window 43 and the calendar file applica- 
tion window 53 in order to transfer the schedule-related 
information. This process was slow, annoying, and presented 
the user with opportunities to make errors in transferring the 
data. As described below, an illustrative embodiment of the 
present invention provides an improved method and system 
for transferring the schedule-related information from the 
e-mail application window 43 to the electronic calendar 
application window 53 

FIG. 5 is a diagram illustrating a user interface including 
an e-mail application window 43 and a window 48 display- 
ing the target data fields of the application-specific template 
in accordance with the present invention. We will refer to the 
window 48 displaying the target data fields of the autosched- 
uler template as the autoscheduler window 48. Referring 
now to FIGS. 1 and 5, the e-mail program 36 and the 
autoscheduler program 38 each use windows to display 
information to a user. The e-mail program 36 presents the 
e-mail message 42 from the e-mail file 41 to a user within 
the e-mail application window 43 displayed on the monitor 
26. The autoscheduler program 38 extracts schedule-related 
information from the e-mail file 41 and presents the 
extracted information to a user within the autoscheduler 
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window 48 displayed on the monitor 26. The windows 43 
and 48 associated with the e-mail program 36 and the 
autoscheduler program 38, respectively, are typically dis- 
played simultaneously. The autoscheduler window 48 

5 includes target fields 50. 52 that correspond to target fields 
56. 58 of the electronic calendar program 40 (FIG. 4). 

The electronic calendar program 40 controls the calendar 
file 54. The autoscheduler program 38 automatically stores 
the information extracted from the e-mail file 41 in the 

10 calendar file 54 without the user having to manually launch 
the electronic calendar program 40. opens the calendar file 
54, and manipulates the electronic calendar application 
window 53. The user may launch the electronic calendar 
program 40. open the calendar file 54. and refer to or 

15 manipulate electronic calendar application window 53 at a 
later time. 

By presenting the application windows 43 and 48 
simultaneously, the autoscheduler program 38 makes it 
easier to extract schedule-related information from a source 

20 file, such as the e-mail file 41. and store the extracted 
information in a target file, such as the calendar file 54. 
Moreover, the autoscheduler program 38 automatically 
interprets data within the e-mail file 41 and makes best 
guesses as to the information that should be extracted from 

25 the e-mail file 41. The extracted terms are incorporated into 
the autoscheduler template and displayed in the autosched- 
uler window 48 as a best guess set of target field assign- 
ments. The user may complete and/or correct and then save 
the set of target field assignments displayed in the 

30 autoscheduler window 48 in the calendar file 54 in a 
predefined manner. Thus, the user need not launch the 
electronic calendar program 40 or open and manipulate the 
calendar file 54 in the process of interpreting and transfer- 
ring data from the e-mail file 41 to the calendar file 54. 

35 FIG. 5 shows a graphical user interface for the autosched- 
uler program 38. in which the e-mail application window 43 
and autoscheduler window 48 are displayed side-by-side. 
The e-mail application window 43 is a semi-structured 
application window that includes a structured area 44 and a 

40 free-text area 46. The autoscheduler window 48 includes a 
structured area 50 and a free-text area 52. The autoscheduler 
program 38 interprets and extracts schedule-related infor- 
mation from the e-mail file 41 corresponding to both the 
structured area 44 and the free-text area 46 of the e-mail 

45 application window 43. The information automatically 
extracted from the e-mail file 41 comprises the best guess of 
the autoscheduler program 38 regarding the information that 
should be stored in the calendar file 54. 
The autoscheduler program 38 typically displays the 

50 extracted best guess information within the structured area 
50 of autoscheduler window 48. The user may manually 
complete and/or correct the information within structured 
area 50 of the autoscheduler window 48. Optionally, the user 
may manually input free-text information into free-text area 

55 52 of the autoscheduler window 48. The information dis- 
played in the autoscheduler window 48 may then be auto- 
matically stored in the calendar file 54 in a predefined 
format. It will be appreciated the windows 43 and 48 shown 
in FIG. 5 illustrate an illustrative embodiment of the present 

60 invention and that many variations may be made to the 
disclosed windows 43 and 48 within the spirit and scope of 
the present invention. 

FIG. 6 shows the e-mail application window 43 and the 
autoscheduler window 48 with more particularity. FIG. 6A 

65 shows the structured area 44 of the e-mail application 
window 43. which includes fields containing predefined 
information. Specifically, the structured area 44 of the e-mail 
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application window 43 includes a "from" target field for goal-driven information interpretation and extraction appli- 

identifying the sender of the e-mail message 42. a "to" target cation program such as the autoscheduler program 38. In 

field for identifying the recipient of the e-mail message 42, step 1100. the computer-implemented process 1000 receives 

a "cc** target field for identifying recipients of copies of the a source file containing free-text information, such as the 

e-mail message 42. a "subject" target field for a short 5 e-mail file 41. Referring to FIG. 6A. the e-mail file 41 

description of the subject of the e-mail message 42. and a includes structured data corresponding to the structured area 

"date** target field for indicating the date on which the e-mail 44 of the e-mail application window 43 and free-text data 

message 42 was sent The e-mail application window 43 also corresponding to the free-text area 46 of the e-mail appli- 

in eludes a free-text area 46. which typically contains free- cation window 43. The e-mail file 41 typically includes 

text information created by the author of the e-mail message 10 schedule-related message terms, some of which may corre- 

42. Information including message terms input by the author spond to the structured area 44 of the e-mail application 

of the e-mail message 42 typically fills the free-text area 46 window 43. But most of the schedule-related message terms 

and the fields of the structured area 44. The autoscheduler are typically embedded within the free-text data cone spond - 

program 38 interprets the information within, and extracts ing to the free-text area 46 of the e-mail application window 

schedule-related information from, both structured area 44 1 5 43. 

and free-text area 46 of the e-mail application window 43. In routine 1200. the goal of the information interpretation 

FIG. 6B shows the structured area 50 of the autoscheduler and extraction system is ascertained. For the disclosed 

window 48 which displays the target fields of the autosched- autoscheduler 38. the goal is to extract schedule-related 

uler template. The autoscheduler window 48 may also information from the e-mail file 41 and to store the extracted 

include a free-text area 52. Each field of the structured area 20 information in the calendar file 54. As noted previously, the 

50 may contain schedule-related information to be recorded principles of the present invention could be applied to many 

in a corresponding target field within the calendar file 54. different types of goal-driven information interpretation and 

Specifically, the structured area 50 of the autoscheduler extraction programs, an example of which is the autosched- 

window 48 includes a "date** target field for identifying the uler 38 described in detail herein. Routine 1200 therefore 

date of a meeting, a "time** target field for identifying the 25 allows a user to select among a plurality of goal-driven 

starting and ending times of the meeting, a "place" target information interpretation and extraction algorithms having 

field for identifying the location of the meeting, and an different goals and different target files. Thus, the computer- 

"attendees** target field for identifying the meeting attend- implemented process 1000 must first ascertain the goal for 

ees. These target fields of autoscheduler window 48 have which the user wants to extract information from the source 

corresponding target fields within the calendar file 54 (FIG. 30 file. For example, the computer-implemented process 1000 

4). may infer a goal based on the content of the source file, or 

The autoscheduler program 38 fills the target fields in the the user may designate a goal on a case-by-case basis using 

structured area 50 of the autoscheduler window 48 with a conventional selection device, such as a pull -down menu, 

message terms extracted from the e-mail file 41 that corre- The steps associated with routine 1200 for ascertaining the 

spond to the structured area 44 and the free-text area 46 of 35 goal of the information interpretation and extraction system 

the e-mail application window 43. The autoscheduler pro- are described with more particularity with reference to FIG. 

gram 38 may also interpret and alter e-mail message terms. 8. 

For example, the e-mail message term 'tomorrow** may be In routine 1300. the extractor associated with the target 

interpreted with reference to the date of the e-mail message file is identified and executed. The extractor uses an 

42 and changed into a date in month, day. year format. 40 application-specific rule base with each application-specific 

FIG. 6C shows the electronic calendar application win- template. Each rule base is a system of rules for interpreting 

dow 53. The electronic calendar application window 53 message terms and assigning the message terms to target 

includes a structured area 56 and a free-text area 58. The fields in the application-specific template. For example, the 

free-text area 58 of the electronic calendar application rule base may include rules for interpreting the meaning of 

window 53 typically includes the information from the 45 message terms based on hints, such as punctuation and the 

free-text area 52 of the autoscheduler window 48. Referring relative position of terms in a message. Continuing with the 

back to FIG. 6A, it wu^ be aj^edated mat information to be e-mail and autoscheduler example, the source file is the 

recorded in the structured area 56 of the calendar file 54 e-mail file 41. the target file is the electronic calendar file 54. 

depends on the context of the free-text information in the and the extractor is the autoscheduler program 38. 

e-mail file 41. Interpreting free-text information is a cogni- 50 In routine 1400, the message terms in the source file are 

tive function that is difficult to emulate with programmed interpreted and certain message terms are automatically 

logic. Although the autoscheduler program 38 includes assigned to target data fields in the application-specific 

intelligence for interpreting free-text information, it does not template. For example, continuing with the e-mail and 

always do so with perfect accuracy. Rather, the autosched- autoscheduler example, the e-mail message terms from the 

uler program 38 makes a best guesses (Le., one best guess 55 e-mail file 41 are assigned to target data fields in the 

for each target field of the electronic calendar application autoscheduler template. As noted previously, the autosched- 

window 53) at interpreting the information within the e-mail uler window 48 and the e-mail application window 43 are 

file 41, displays the best guesses to the user in the autosched- typically displayed on the computer monitor 26 simulta- 

uler window 48, and allows the user to complete and/or neously. The target data fields displayed in the window 48 

correct the information within the autoscheduler window 48. 60 are tied to structured fields of the calendar file 54 so that by 

The autoscheduler program 38 also provides the user with entering data into the target data fields of the autoscheduler 

alternative choices for target field entries either by providing window 48. the user specifies an entry to be made in the 

alternative choices for a particular target field selectable via calendar file 54. 

a pop-down menu or for a new set of target fields if available According to one aspect of the present invention, the 

from the analysis of the free-text information. 65 computer-implemented process 1000 automatically makes 

FIG. 7 shows a logic flow diagram for an illustrative best guesses as to the e-mail message terms from the e-mail 

embodiment of a computer-implemented process 1000 for a file 41 that should be included in the target fields of the 
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application-specific template. Thus, the user has a head start 
on transferring information from the source file to the target 
file and need only complete and/or correct the information 
initially assigned to the target data fields of the application- 
specific template displayed to the user by the computer- 5 
implemented process 1000. The steps associated with rou- 
tine 1400 for automatically interpreting and assigning 
message terms to the target data fields in the application- 
specific template are describe with more particularity with 
respect to FIGS. 9-10. 10 

In step 1500. the target data fields of the application- 
specific template is presented to the user for completion 
and/or correction. In the illustrative embodiment, the user 
manually inputs information into the autoscheduler window 
48 or selects from entries made available by the computer- 15 
implemented process 1000 through pull-down menus. Upon 
an indication from the user that the information contained in 
the autoscheduler window 48 is complete and correct, the 
information displayed in the autoscheduler window 48 is 
stored in the calendar file 54. The steps associated with 20 
routine 1500 for allowing the user to interactively complete 
and/or correct the target data fields of the application- 
specific template are described with more particularity with 
reference to FIG. 11. 

In routine 1600. the target data fields of the application- 25 
specific template are saved in the calendar file 54. In 
addition, routine 1600 provides for updating the lexicon of 
the template that was used to extract and assign data to the 
application-specific template. This allows the lexicon of the 
autoscheduler program 38 to be updated with information 30 
learned from manual user input. The steps of routine 1600 
for saving the target data fields of the application-specific 
template are described with more particularity with respect 
to FIG. 12. 

The computer-implemented process 1000 illustrated in 35 
FIGS. 7 through 12 provides a convenient method for a user 
to extract schedule-related information from source files for 
storage in target files. It will be appreciated that a user may 
be interacting with any of a variety of application programs 
on the computer system 8 when the user decides to read his 40 
or her e-mail. At a convenient point in time for reading 
e-mail, the user may select an e-mail icon on a menu bar to 
activate the e-mail program 36. The e-mail program 36 then 
displays an e-mail message 42 in the e-mail application 
window 43 on the computer monitor 26. If the e-mail 45 
message 42 contains schedule-related information, the user 
may quickly activate the autoscheduler program 38 in a 
conventional manner, for example by clicking an icon on a 
menu bar. This simple action invokes the autoscheduler 
program and, if necessary, displays the autoscheduler win- 50 
dow 48 to simultaneously appear on the screen with the 
e-mail application window 43. 

Upon activation of the autoscheduler program, the target 
data fields of autoscheduler template arc automatically 
populated with best guesses as to which message terms from 55 
the e-mail file 41 should be included in the target calendar 
file 54. The user may quickly complete and/or correct the set 
of target field assignments in the autoscheduler window 48 
and saves the results. In this manner, the user quickly 
transfers information from the e-mail file 41 to the target 60 
calendar file 54. The user may then return to the application 
program the user was interacting with when the user decided 
to read his or her e-mail. 

FIG. 8 is a logic flow diagram illustrating the steps of 
routine 1200 for ascertaining the goal of the automatic 65 
information interpretation and extraction system. Those 
skilled in the art will appreciate that the decision engine 208 
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may use alternatively probabilistic reasoning such as Baye- 
sian inference or logical reasoning employing a rule base to 
analyze the system context and user commands to determine 
which goals of a set of system defined goals are unambigu- 
ously satisfied. In step 1202, the decision engine 208 deter- 
mines if there is at least one unambiguous goal satisfied. If 
there are no goals determined to be unambiguously satisfied, 
then a pop- up dialog window is presented to the user to 
identify the goal intended by the user at step 1212. The goal 
selected by the user is the goal ascertained at step 1214. 
Where the decision engine 208 has identified only a single 
goal to be unambiguously satisfied at step 1204, then that 
goal is selected at step 1214 as the ascertained goal 

If the decision engine 208 has unambiguously determined 
that more than one goal has been satisfied, then the decision 
engine performs an inference analysis at step 1206 using 
probabilistic reasoning or alternatively logical reasoning to 
analyze system context and user commands as discussed 
above. The inference results of step 1206 are input to a 
cost-benefit analysis performed at step 1208 to determine 
which goals exceed defined utility measures. If the decision 
engine 208 at step 1210 does not find at least one goal that 
exceeds it's defined utility measure as a result of the 
cost-benefit analysis performed at step 1208. then a pop-up 
dialog window is presented to the user to identify the goal 
intended by the user at step 1212. The goal selected by the 
user is the goal ascertained at step 1214. Where there is at 
least one goal that exceeds it's defined utility measure as a 
result of the cost-performance analysis performed at step 
1208. then the decision engine 208 selects the first goal at 
step 1214 as the ascertained goal. After step 1214. the 
computer-implemented process 1000 continues with routine 
1300. 

In an alternative embodiment, when the decision engine 
208 has unambiguously determined at step 1204 that more 
than one goal has been satisfied, steps 1206. 1208 and 1210 
are skipped and a pop-up dialog window is presented to the 
user to display an alphabetical list of goals for the user at 
step 1212. It will appreciated that routine 1200 provides 
flexibility for different users to select different goals. This 
allows users to tailor the operation of the computer- 
implemented process 1000 to their particular needs. This 
aspect of the disclosed embodiment of the present invention 
makes the computer- implemented process 1000 useful for a 
wide variety of users. 

In the illustrative embodiment disclosed, the autosched- 
uler program 38 uses a set of simple rules to ascertain 
whether the goal of extracting scheduling information from 
an e-mail message for automatically scheduling an appoint- 
ment in a calendar file is unambiguously determined. For 
example, as a first rule, if e-mail is open and the user enters 
a special keyboard command. Alt-A. then the goal of 
extracting scheduling information from the e-mail message 
is unambiguously determined to be the goal intended by the 
user. As a second rule, when an e-mail message is selected 
and dragged to the calendar icon, the goal of extracting 
scheduling information from the e-mail message is unam- 
biguously determined to be the goal intended by the user. 
Alternatively, the user may select a goal, for example by 
selecting an item in a menu bar or in a pull-down menu. The 
user reading the e-mail message may realize after reading 
the e-mail message that the message contains information 
that the user wants to store in a particular target file. The user 
may then click on an icon in a menu bar to access a 
pull-down menu or otherwise input a command signal such 
as a keyboard command sequence to indicate the target file 
or that the associated extractor such as the autoscheduler 
should be run. 
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FIG. 9 illustrates the steps of routine 1400 that the higher numbers of expected terms. This ranking method also 

extractor 210 employs when executed at step 1300 of FIG. assigns a higher priority to clusters that include terms that 

7 for automatically assigning data from the source object to are closer together. Specifically, the following formula may 

the application-specific template. In the illustrative be used to rank clusters: 

embodiment, the source object is the e-mail file 41. and the 5 
application-specific template is the autoscheduler template. 
Routine 1400 uses the autoscheduler template for assigning 
message terms from the e-mail file 41 to the target data fields 
the calendar file 54. Routine 1400 also ranks the candidate 

terms for assignment to the target data fields in a priority 10 _i 

order. In step 1402. the template for extracting information t 2 
from the source object is retrieved from memory. In the 

illustrative embodiment, the autoscheduler template where k is me nurabff of relevant terms in the cluster and a^ 

includes a group of 'tokens" that correspond to target data is me num0C r of message terms from the relevant term(i) to 

fields in the electronic calendar file 54. Each token corre- 15 the beginning of the e-mail file. The exponent "2" may be 

sponds to a group of expected terms for which the computer- varied. In step 1444. the cluster of tagged terms with the 

implemented process 1000 searches in the source file. The highest priority is selected for assignment to target fields in 

autoscheduler uses a rule base for interpreting message the application-specific template. 

terms, tagging expected terms found in the e-mail message In an alternative embodiment rather than building a set of 

with a token, and assigning the tagged words to target fields 20 fields and then scoring the set of fields as described above in 

in autoscheduler template. reference with FIG. 10. each target data field of the template. 

In step 1410. the autoscheduler program tags expected all the candidates for a single field can be ranked and scored 

terms appearing within the e-mail 41 with their respective before proceeding to the next field. Hie order in which the 

tokens. For example, the autoscheduler may use the template fields have candidates ranked and scored may vary based 

to tag the terms 'today." 'tomorrow," "next Friday," etc. 25 upon the source object and the target object 

with the token "day word.** The "day word** token corre- An exemplary method is described for automatically 

sponds to the "date" target data field in the electronic generating and refining a scoring function which may be 

calendar file under which the extracted information will used to rank candidate terms in priority order. The first step 

ultimately be stored. Known irrelevant terms are automati- is to acquire a set of source objects from which data objects 

cally skipped. Known irrelevant terms typically include the 30 will be extracted and applied to an application-specific target 

legends of the structured fields of the e-mail application object. For the illustrative embodiment, the set of source 

window 43, such as 4 W "from," "cc.** "subject.** and objects are a set of e-mail messages containing scheduling 

"date." These terms cannot comprise part of the schedule- information. The next step is to identify a set of distinction 

related information and are therefore excluded from further in the content of the source objects that express some 

consideration as expected terms. These terms may, however. 35 structural relationship to candidate terms of interest which 

provide useful hints for interpreting other message terms. will be applied to the target object These distinctions may 

Once the expected terms of the e-mail message are include: the occurrence of special words within words 

tagged, at step 1420 the candidate tagged terms are ranked before a candidate term(s) of interest; the occurrence of 

in priority order for assignment to the target data fields of the special words within words after candidate terms(s) of 

electronic calendar file 54. An illustrative process for rank- 40 interest; the occurrence of special words somewhere in the 

ing candidate tagged terms in priority order for assignment document before a candidate term of interest; the occurrence 

to the target data fields of the autoscheduler template is of special words somewhere in the document after a candi- 

described with more particularity with reference to FIG. 10. date term of interest; the general location of the candidate in 

At step 1446. the candidate tagged terms with the highest the document (e.g.. in the first half or second half of the 

priority are assigned to the target data fields of the 45 document); the location of other candidates in the document 

autoscheduler template. In step 1448. the autoscheduler with regard to a particular candidate; and the particular form 

window 48 including the best guess set of target field of the candidate (e.g.. an abstract reference such as the 

assignments is displayed to the user. The computer- following for referring to a particular date — 'today," 

implemented process 1000 then continues with routine 1500 'tomorrow," or an abbreviation for a particular day). 

(FIG. 11). 50 After a set of distinctions have been identified that express 

FIG. 10 illustrates an exemplary process for ranking some structural relationship to candidate terms of interest, a 

candidate tagged terms in priority order for assignment to set of target objects that have been correctly instantiated 

the target data fields of an application-specific template. In manually are selected to gather probabilistic information for 

step 1422. the tagged terms are grouped into clusters to build each term on the relationships between the observations of 

sets of candidate terms for assignment to the template target 55 distinctions and the correct candidates. This information can 

fields. A tagged term is grouped into a cluster based on its include the probability that a distinction is found to be 

spatial relationship to other tagged terms in the source file. present given that a term is the correct term for the field in 

For example, all tagged terms that are within four message the application-specific template. For example, considering 

terms of another tagged term may be grouped into the same the time of a meeting in the illustrative embodiment of the 

cluster. 60 autoscheduler. data is collected on the probability 

In step 1430. the clusters are ranked in a priority order. It p( distinction x see correct date y) where distinction x can be 

has been discovered that clusters that contain more relevant the word "at" seen adjacent to a term tagged as a candidate 

terms are generally more likely to include important time. At runtime. Bayes rule is applied to take a set of such 

information, and that clusters in which the relevant terms are probabilities for a variety of distinctions and compute the 

more tightly grouped are generally more likely to include 65 approximate probability of the correct date given multiple 

important information. One method for ranking the clusters distinctions. p(correct date y distinctions 1 . . . n), for each 

therefore assigns a higher priority to clusters that contain a candidate time. 
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FIG. 11 shows the steps of routine 1500 for obtaining user template to die target object In decision step 1602. it is 

input to modify and/or complete a set of target data field determined whether the template lexicon should be updated, 

assignments for the application-specific template. Referring Typically, a user specifies when the template lexicon should 

to FIG. 9, in step 1448. the best guess set of target data field be updated on a case-by-case basis. This allows the user to 

assignments to the appncation-specific template are dis- 5 identify changes made to a particular template that the user 

played within the autoscheduler window 48 on the monitor w ^ s to be permanently stored as part of the template 

26. In step 1502. the computer-implemeQted process 1000 l ™con : If the template lexicon is to be updated, the ^ts 

gets a user command, typically either a save command or a ^ foUowed from step 1602 to step 1604.* which the 

command to modify the set of target field assignments £ mp *f /*TV S ^ 

... .... w j i ■ a AO i a • • .« be updated, the no branch is followed from step 1602 to 

dismayed in the autoscheduler window 48. In decision step 10 ^ Stc 1604 h foUowed b stc 1606 m ^ mc 

1504. it is determined whether the user command is a save ^ me fe For example! referring to the illustra- 
command. If the user command is a save command, the tjve embodiment of the autoscheduler. the calendar file 54 is 
"yes" branch is followed from step 1504 to the continue updated witn the information included in the application- 
step, in which the computer-implemented process 1000 specific template 48. 

continues with routine 1600 (FIG. 12). If the user command 15 The preferred templates and rules for implementing an 

is not a save command, the "no" branch is followed from autoscheduler according to the present invention are 

step 1504 to step 1506. in which the edit mode is set. included in the "Attachment 1" which is appended hereto. 

There are three exemplary modes for editing the target it should be understood that the foregoing description and 

field assignments of the application-specific template.: (1) "Attachment 1" relate only to the illustrative embodiment of 

the direct edit mode. (2) the target field select mode, and (3) 20 the present invention, and that numerous changes may be 

the global candidate field set mode. The direct edit mode made therein without departing from the spirit and scope of 

allows the user to manually edit the information on a the invention as defined by the following claims. Because of 

per-target-field basis. The target field select mode allows the the variations that can be applied to the illustrated and 

user to select alternatives using a pull-down menu on a described embodiments of the invention, the invention 

per-target-field basis. The global candidate set field mode 25 sh <^ d ■* solcl y with reference to the appended 

allows the user to select a new cluster of candidate fields claims* 

when more than one cluster exists. **** is ddmcd is: 

In decision step 1508. it is determined whether edit mode 1 A ^thod for extracting information from a source 

is set to the direct edit mode. If the edit mode is not set to ™ d * e ntxutod information 10 a 

the direct edit mode, the "no" branch is followed from 1508 30 ob i^ comprising the steps of: 

back to 1506. If the edit mode is set to the direct edit mode, selecting a source object with untagged informatioQ; 

the ik y esM branch is followed to step 1510. in which the selecting a target object that stores data objects extract- 

autoscheduler window 48 is modified in response to manual able from the untagged information of the source 

user edit commands. In the direct edit mode, the user simply object, 

types the desired data directly into the target fields of the 35 interpreting free-text information within the source file; 

autoscheduler window 48. based on the interpretation, extracting data objects of the 

In decision step 15 1Z it is determined whether the edit type that are storable in the target object from the 

mode is set to the target field select mode. If edit mode is not untagged information of the source object; and 

set to the target field select mode, the "no" branch is storing the data objects in the target object, 

followed from step 1512 back to 1506. If the edit mode is set 40 2. The method of claim 1 wherein the step of interpreting 

to the target field select mode, the **yes" branch is followed free-text information within the source file further comprises 

from step 1512 to step 1514. in which alternatives for a the steps of: 

particular target field of the autoscheduler template are accessing a template of data patterns, each data pattern 

displayed. Topically, the target field select mode works with associated with one or more of the data objects to 

a pull-down menu that displays a set of alternatives for a 45 bc storcd m mc object; 

particular target field in the application-specific template. In searching for occurrences of each data pattern within the 

step 1510, one of the alternatives from the pull-down menu source obiect* 

is seated in response to user commands. togging cadi ocaurencc of each date pattern found within 

In decision step 1516, it is determined whether the edit the source obect* 

mode is set to the global candidate field set mode. If the edit 50 . . , , c ^ 

mode is not set to the global candidate field set mode, the assigning each tagged data pattern to at least one of the 

"no" branch is followed from step 1516 back to step 1506. ^J 8 ™* 

If the application window edit is set to the global candidate rankiD g thc <*>P cts m a P™ 11 * order ' r t . 

field set mode, the "yes" branch is followed from step 1516 3 * ***** "^thod of claim 2 whereiD the sle P of rankui S *f 

to step 1518. In step 1518. the next most likely set of 55 <*>i tcts in a P™ 1 ** ordcr furthcr co^P^s * c slc P sof: 

candidate fields as determined by ranking step 1430 is applying a Bayesian inference engine to rank the data 

displayed. After steps 1518 and 1510. routine 1500 loops objects by likelihood of their correctness; 

back to step 1502. in which the computer-implemented grouping the tagged target data objects into a plurality of 

process 1000 receives another user command. It will be clusters; and 

appreciated that routine 1500 loops through steps 1502 60 ranking the clusters in a priority order. 

through 1518 until the user is satisfied with the entries in the 4. The method of claim 1 wherein the step of selecting a 

application-specific template and inputs a save command. source object further comprises the step of receiving a user 

Another command for exiting routine 1500. such as an abort command from a user input device selecting the source 

or exit command, may be provided to exit this routine object. 

without saving the results. 65 5. The method of claim 1 wherein the step of selecting a 

FIG. 12 shows the steps of routine 1600 for saving the target object further comprises the step of applying an 

target data field assignments of the application-specific inference engine to select the target object. 
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6. The method of claim 1 wherein the step of selecting a 
target object further comprises the step of receiving a user 
command from a user input device associating the target 
object with the source object 

7. A method for interpreting and extracting terms from a 5 
free-text portion of a source file, comprising the steps of: 

retrieving the source file; 

retrieving a application-specific template comprising one 
or mote target fields for accepting assignments, the 
target fields being associated with data items to be 10 
recorded in a target file; 

retrieving a template comprising a plurality of expected 
terms, each expected term being associated with one or 
more target fields of the application-specific template; 

tagging terms, within the free-text portion of the source 
file, that correspond to one or more of the expected 
terms; 

grouping the togged terms into a plurality of clusters; 
ranking the clusters in a priority order; and 20 
assigning the tagged terms to the target fields in the 
priority order. 

8. The method of claim 7. wherein: 

the application-specific template is selected from a plu- 
rality of predefined application-specific templates 25 
based on a goal inferred from data within the source 
file; 

- the template is selected from a plurality of predefined 
templates based on the goal inferred from data within 
the source file; and 30 
the target file is selected from a plurality of predefined 
target files based on the goal inferred from data within 
the source file. 

9. The method of claim 7, further comprising the steps of: 35 
presenting the application-specific template including the 

target field assignments to a user for review and modi- 
fication; and 

in response to a user command, storing the target field 
assignments in the target file. 40 

10. The method of claim 9. further comprising the step of: 
in response to a user command, presenting a list of 

alternative assignments for one or more of the target 
fields. 

11. The method of claim 10, further comprising the step 45 

of: 

in response to a user command, changing the assignment 
for one or more of the target fields. 

12. The method of claim 11. wherein alternative target 
field assignments are displayed in a pop-up window on a 50 
monitor. 

13. The method of claim 9. wherein the source file 
comprises terms in structured fields. 

14. The method of claim 13. wherein the source file is an 
electronic mail file. 55 

15. The method 13. wherein the target file is an electronic 
calendar file. 

16. The method of claim 11. further comprising the step 
of updating the template to reflect the changed target field 
assignment 60 

17. A computer-readable medium storing a program mod- 
ule comprising instructions which, when executed by a 
computer system comprising a processing unit, a memory 
storage device, and a user input/output system including a 
monitor, cause the computer system to interpret and extract 65 
terms from a free-text portion of a source file by performing 
the steps of: 
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retrieving the source file; 

retrieving an application-specific template comprising 

one or more target fields for accepting assignments, the 

target fields being associated with data items to be 

recorded in a target file; 
retrieving a template comprising a plurality of expected 

terms, each expected term being associated with one or 

more target fields of the application-specific template; 
tagging a set of terms, within the free-text portion of the 

source file, that correspond to one or more of the 

expected terms; 
grouping the tagged terms into a plurality of clusters; 
ranking the clusters in a priority order; 
assigning the tagged terms to the target fields in the 

priority order; and 
displaying the application-specific template including the 

selected target field assignments on the monitor for 

review and modification. 

18. A computer-readable medium storing a program mod- 
ule comprising instructions which, when executed by a 
computer system coniprising a processing unit, a memory 
storage device, and a user input/output system including a 
monitor, cause the computer system to interpret and extract 
terms from free-text information by performing the steps of: 

displaying a first window on the monitor; 

displaying a portion of the free-text information within 

the first window; 
inferring a goal based on the free-text information; 
in response to the goal. 

selecting a second window from a plurality of 
application-specific templates, the second window 
comprising one or more target fields for accepting 
assignments, the target fields being associated with 
data items to be recorded in a target file, and 
retrieving a template from a plurality of predefined 
templates, the selected template comprising a plu- 
rality of expected terms, each expected term being 
associated with one or more target fields of the 
application-specific template; 
tagging a set of terms, within the free-text information, 
that correspond to one or more of the expected terms; 
grouping the tagged terms into a plurality of clusters; 
ranking the clusters in a priority order; 
assigning the tagged terms to the target fields of the 

second window in the priority order; 
displaying the second window on the monitor; and 
displaying the assigned terms on the monitor within the 
target fields of the second window. 

19. The computer-readable medium of claim 18. wherein 
the first and second windows are displayed simultaneously 
on the monitor. 

20. A computer-readable medium storing a program mod- 
ule comprising instructions which, when executed by a 
computer system comprising a processing unit, a memory 
storage device, and a user input/output system including a 
monitor, cause the computer system to perform the steps of: 

recognizing a user gesture; 

ascertaining a user goal associated with the user gesture; 
identifying goal-related source information in one or more 

source objects; 
interpreting free-text information within the source file; 
based on the interpretation determining a relevant portion 

of the source information; 
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extracting the relevant portion of the source information; 
identifying one or more target objects; and 
altering the target objects by applying the extracted infor- 
mation to the target objects. 

21. The computer-readable medium of claim 20, wherein 
the step of ascertaining a user goal further comprises the step 
of inferring a user goal based on a predefined system context 
and a pattern of user commands received during a current 
user interface session. 

22. The computer-readable medium of claim 20. wherein 
the instructions further cause the computer to perform the 
steps of: 

before the step of determining a relevant portion of the 
source information, 

presenting on the user input/output system a query 
whether to proceed with a proposed data extraction. 
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receiving a user command to proceed with the proposed 

data extraction, and 
retrieving the source objects from the memory storage 

device. 

23. The computer-readable medium of claim 22, wherein 
the instructions further cause the computer to perform the 
steps of: 

after the step of altering the target objects by applying the 
extracted information to the target objects. 
) presenting on the user input/output system a query 
whether to proceed with a proposed data storage, 
receiving a user command to proceed with the proposed 

data storage, and 
storing the altered target objects in the memory storage 
. device. 

* * * * * 
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